Repository navigation
[mosaic] Reject TIMESTAMP(0) when validating Mosaic data fields - #11241
LuciferYang wants to merge 1 commit into
Conversation
|
This addresses the same writer limitation in the same schema-validation path as the earlier #11235. I verified that the two changes reject the same precision-zero types, including when the visitor recurses into a collection. The 162 Mosaic tests passed on the merge with current master, including the new nested cases. Closing this duplicate so #11228 can be resolved in one PR. Please carry the nested ARRAY/MAP cases into #11235. One upgrade scenario matters for the retained fix. I wrote a row to a Mosaic table and added a nullable TIMESTAMP(0) column using master. Master still returned the existing row with NULL in the new column. With this patch, that same read fails during table initialization, before opening any data file. The retained fix should reject unsupported new definitions or writes while preserving reads of existing data. |
|
Thank you @JingsongLi |
Purpose
close #11228
Precision 0 maps to the Arrow
SECONDunit, which the Mosaic native writer does not accept. A Mosaic table with aTIMESTAMP(0)orTIMESTAMP(0) WITH LOCAL TIME ZONEcolumn can be created, but every write fails withwriter open failed: unsupported Timestamp unit: Second. MySQL CDC creates such columns fromDATETIME.MosaicFileFormat.validateDataFieldsnow rejects precision 0, also insideARRAYandMAP, with the same message style as the other types Mosaic rejects, and suggestsTIMESTAMP(3).This check also runs when a table is loaded. An existing Mosaic table with a
TIMESTAMP(0)column could not be written before and now cannot be read either, until the column is changed toTIMESTAMP(3)orfile.formatis set back. Both ALTERs work on such a table.Tests
MosaicFileFormatTest:testValidateDataFieldsTimestampPrecisionZeroUnsupportedchecks the full message forTIMESTAMP(0),ARRAY<TIMESTAMP(0)>,TIMESTAMP(0) WITH LOCAL TIME ZONEandMAP<STRING, TIMESTAMP(0) WITH LOCAL TIME ZONE>. It fails without the change.testValidateDataFieldsTimestampPositivePrecisionSupportedchecks that precisions 1 and 9 of both types are still accepted.